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ABSTRACT 



This report is part of a series of special topic reports 
developed by the Chicago Annenberg Research Project to document key issues 
and problems affecting the Chicago Annenberg Challenge and the improvement of 
Chicago public schools in general. The report introduces the concept of 
instructional program coherence and explains why schools whose improvement 
efforts show strong coherence are more likely to advance. It presents 
research showing that students in Chicago elementary schools with stronger 
program coherence have higher gains in academic achievement. It also shares 
observations on how, in specific schools, principals, external partners, and 
other agencies direct, or fail to direct, key school resources toward more 
coherent instruction. It ends by discussing factors within the educational 
system that discourage instructional program coherence, suggesting ways that 
school leaders, school improvement partners, and policymakers can bring about 
instructional coherence that will reward their school improvement efforts. 
Five chapters present: (1) "The Problem: Too Many Unrelated, Unsustained 

'Improvement' Programs"; (2) "What Is Instructional Program Coherence?" (3) 
"Is Instructional Program Coherence Related to Student Achievement: Survey 
Results"; (4) "Organizing Instructional Program Coherence in Schools: Field 
Study Results"; and (6) "Interpretive Summary." Overall, research shows that 
schools ranking high on instructional program coherence have stronger 
principal leadership, while lower ranking schools allow teachers more 
individual autonomy and the discretion to select their own curriculum 
materials and strategies for instruction and assessment. An appendix presents 
the statistical analysis. (Contains 43 references.) (SM) 
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A large array of urban school improvement grants, 

Z-\ programs, and partnerships enables schools to 
-A JL. embrace multiple improvement initiatives in an 
effort to improve student achievement. This report dis- 
cusses an important reason why schools involved in 
many reform efforts do not always improve their stu- 
dents' achievement. 

We introduce the concept of instructional program co- 
herence and explain why schools whose improvement ef- 
forts show strong coherence are more likely to advance. 
We present new evidence showing that students in Chi- 
cago elementary schools with stronger program coherence 
show higher gains in student achievement. We also share 
observations on how, in specific schools, principals, exter- 
nal partners, and other agencies direct, or fail to direct key 
school resources toward more coherent instruction. In clos- 
ing, we discuss factors within the educational system that 
discourage instructional program coherence, and sug- 
gest ways in which school leaders, school improvement 
partners, and policy makers can act to bring about the 
instructional coherence that will reward their school im- 
provement efforts. 
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T r n 1993, Ambassador Walter Annenberg announced a $500 mil- 
lion challenge grant to improve public education in the United 
J_L States. Cities wishing to receive a portion of that grant were 
invited to submit proposals describing how the funds would be 
used to stimulate educational innovation and collaboration in their 
public school systems. A group of Chicago school reform activists 
and education stakeholders, including parents, teachers, principals, 
community leaders, and foundation officers, organized to write a 
proposal to include Chicago among the sites receiving a grant. They 
were successful. In January 1995, the Annenberg Foundation 
awarded a five-year grant of $49.2 million to establish the Chicago 
Annenberg Challenge. An additional $100 million in matching 
funds was pledged by local donors. 

The Chicago Annenberg Challenge was organized to distribute and 
manage these monies among networks of schools and external part- 
ners throughout the city. Its mission is to improve student learning by 
supporting intensive efforts to reconnect schools to their communi- 
ties, restructure education, and improve classroom teaching. The Chi- 
cago Challenge funds networks and external partners that seek to 
develop successful, community-based schools that address three criti- 
cal education issues through whole-school change: school and teacher 
isolation, school size and personalism, and time for learning and im- 
provement. More than half of Chicago’s public schools will have par- 
ticipated in an Annenberg-supported improvement effort by the end 
of the grant period in 2001. 

This report is part of a series of special topic reports developed by 
the Chicago Annenberg Research Project. This series focuses on key 
issues and problems of relevance to the Chicago Annenberg Challenge 
and to the improvement of Chicago public schools generally. It comple- 
ments a series of technical reports that focus specifically on the work 
and accomplishments of the Chicago Annenberg Challenge. Among 
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the topics examined to date in the special topics 
report series are the quality of intellectual work in 
Chicago elementary schools; social support, aca- 
demic press, and their relationship to student 
achievement; and, in this report, school instruc- 
tional program coherence. 

The work of the Chicago Annenberg Research 
Project is intended to provide feedback and useful 
information to the Chicago Challenge and the schools 
and external partners who participate in its efforts to 



improve educational opportunities for Chicago’s chil- 
dren and youth. This work is also intended to expand 
public discussion about the conditions of education 
in the Chicago Public Schools and the kinds of ef- 
forts needed to advance meaningful improvements. 
This effort to stimulate new avenues of discussion 
about urban school improvement is an important as- 
pect of Ambassador Annenberg s challenge to engage 
the public more fully in school reform. 
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A TV newscaster, on assignment to publicize progress in urban 
school reform, is explaining why he selected Travis Elementary 
.School for his story : 1 

“There’s so much happening. The teachers are a great group and 
their principal is always getting them new programs and equip- 
ment. The entryway has plaques, commendations, pictures and 
letters to show all the activities they are involved in. It lists all the 
school improvement initiatives, too. For reading alone they have 
Reading Recovery, SRA, Great Books, Accelerated Reader, Drop 
Everything and Read (DEAR), and Links to Literacy. The math- 
ematics initiatives include Plato, The Algebra Project, Family 
Math, and a new textbook series. The school has programs with 
over a dozen social service partners and community groups. Lots 
of the faculty sponsor special programs for the kids, too. Gosh, I 
remember when I used to visit these urban schools and they 
seemed like graveyards where nothing was happening and few 
were trying.” 

The reporters positive comments were appreciated, but people more 
familiar with Travis knew that, in spite of all the hard work and innova- 
tion, student scores in reading and mathematics remained far below the 
districts performance standards. The principal and teachers attended meet- 
ings and conferences to take advantage of new opportunities, but many 
staff members were tired and frustrated. One teacher said: 

“Some Saturday mornings I can t remember which workshop Im 
supposed to go to. I know its a bad weekend when Im supposed 
to be at two at once. Or, when what they tell us to do at one 
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workshop is the opposite of what was suggested 
at the last. In class, I just try out bits of each as 
best I can.” 

Travis’ principal respected her teachers and worked 
tirelessly to bring them new resources and ideas: 

“My teachers are doing their absolute best. 
They’re trying terribly hard, most of them. 
There’s so much out there to do. And I keep 
finding more, and asking them to do more. 
Sometimes I feel like we’re juggling too many 
balls in the air, and either we’re going to drop 
the balls, or we’re going to be so tired, and pulled 
in so many directions, we’re the ones who are 
going to drop.” 

The story of Travis reflects a trend among many 
urban schools, including schools in Chicago. On the 
one hand, administrators and teachers want to adopt 
programs and materials that might help them teach 
more effectively. On the other, there are so many 
meetings and workshops, that staff express not only 
fatigue, but also professional frustration. They find 
themselves faced with a large and fragmented array 
of school improvement grants, programs, and part- 
John Booz 



nerships that rarely afford them the time or support 
to adopt and master practices that may improve stu- 
dent learning. 

Principals are caught in a bind. Some recognize 
that their faculty might be “juggling too many balls.” 
On the other hand, they also know that they cannot 
expect their schools to improve if everything stays the 
same. Hooking up with multiple improvement ini- 
tiatives often seems the only way to bring needed re- 
sources to a school and to promote the work and 
commitment of staff with very different interests or 
strengths. Moreover, the emotional and social needs 
of many of their students are enormous, far beyond 
what a school can address on its own. For this reason, 
partnerships with external organizations seem essen- 
tial. Even when principals recognize that their teach- 
ers are stretched in too many directions, they seem 
unable to cut programs, believing strongly that they 
need all of these extra resources to help the children. 

Research has documented the importance of school 
organizational factors such as a unity of purpose, a 
clear focus, and shared values for student learning. 
Research has also drawn attention to the problem of 
incoherent school programs, where diverse initiatives 
set up to serve important needs, but which lack the 
sustained attention of the majority of staff within the 
school, have no apparent effects on the core 
goal of improving student achievement. 
Earlier Consortium research focused spe- 
cific attention on the problems of “Christ- 
mas tree” innovations; that is, change or 
improvement strategies that bring attention 
to a school through numerous program 
and equipment purchases but fail to build 
its capacity to improve teaching and learn- 
ing . 2 Other research has pointed out how 
cluttered and contradictory state and dis- 
trict policy environments also fragment 
school development efforts. All this im- 
plies the need for instructional program 
coherence . 3 

Research has not, however, addressed 
how administrators and teachers might ac- 
tually rethink their circumstances and bring 
greater coherence to their efforts. For ex- 
ample, research into school improvement in 
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Chicago has highlighted the importance of five essential 
supports to guide school improvement: school leader- 
ship, parent and community involvement, professional 
community, a student-centered learning climate, and 
high quality instruction . 4 Schools can, however, approach 
the development of these supports in more or less coor- 



dinated ways, and each support could be present with- 
out strong instructional program coherence. To ad- 
dress this issue, we offer a clear operational definition 
for instructional program coherence and we illustrate 
how some schools in Chicago achieved greater coher- 
ence than others. 



School Instructional Program Coherence 7 
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S trong program coherence is evident when three major condi- 
tions prevail in a school. 

1. A common instructional framework guides curricu- 
lum, teaching, assessment, and learning climate. This 
framework combines specific expectations for student 
learning with specific strategies and materials to guide 
teaching and assessment. 

One example of this, observed in Annenberg school sites, is the 
Cunningham approach to literacy. This framework provides grade- 
aligned curriculum, instructional strategies, and assessments. It speci- 
fies that all literacy learning activities be organized into four areas: 
guided reading, self-selected reading, working with words, and writ- 
ing . 5 Another example is the Success for All approach to mathematics 
instruction which provides grade-aligned curriculum, makes learning 
expectations clear (for example, students must engage in mathemati- 
cal reasoning in situations beyond school), and specifies effective in- 
structional strategies for particular types of learning, such as 
manipulatives to represent concepts or symbols, and cooperative learn- 
ing methods for problem solving . 6 Schools can, of course, develop their 
own instructional frameworks, rather than adopting one from an ex- 
ternal source. 

More specifically, 

• Curriculum, instructional strategies, and assessments of students 
are coordinated among teachers within a grade level. 

• Curriculum and assessments of students proceed logically from 
one grade level to the next and offer a progression of increasingly 
complex subject matter rather than repeating rudimentary mate- 
rial previously taught. 



• Key student support programs, such as tutoring, 
remedial instruction, parent education, and op- 
portunities for parental involvement, are aligned 
with the schools instructional framework. 



2. Staff working conditions support imple- 
mentation of the framework. 

• Administrators and teachers expect one another 
to implement the framework. 

• Criteria for recruiting and hiring teachers empha- 
size commitment to and competence in execut- 
ing the framework. 

• Teachers are evaluated and held accountable in 
large part on the extent to which they effectively 
use the common instructional framework. 

• Professional development opportunities for 
staff are focused on the common instructional 
framework, and are pursued over a sustained 
period of time . 7 



3. The school allocates resources such as 
materials, time, and staff assignments to 
advance the school’s common instruc- 
tional framework and to avoid diffuse, 
scattered improvement efforts. 

• Curriculum and student assessments remain 
stable over time. 

• Similarly, teachers’ professional assignments are 
stable enough, so that teachers have sustained op- 
portunities to learn how to teach well in their spe- 
cific roles. 

The concept of coherence has surfaced in many 
different discussions of school improvement, most 
notably on issues of curriculum. Calls for coherent 
curriculum ask primarily for sensible connections 
among the topics that students study in each subject 
within a grade and as they advance through the grades. 



Instructional program coherence includes curricular 
coherence, but entails several other criteria as well. 

Coherence has also been interpreted as alignment 
of the schools instructional program with external 
policies and standards. Our definition does not stipu- 
late this type of alignment, because external policies 
or mandates could promote, undermine, or have no 
effect on the degree of instructional program coher- 
ence within a school. Whether alignment with exter- 
nal policy promotes instructional coherence within a 
school depends upon the nature of the external poli- 
cies. For example, as we look across the country we 
see numerous instances where school alignment with 
the following policies might exacerbate program in- 
coherence: (a) the state mandates curriculum stan- 
dards that call for student mastery of hundreds of 
discrete competencies with no common themes or 
skills connecting them; (b) the district recommends 
that all elementary teachers participate in one-day 
workshops on portfolio assessment, classroom man- 
agement, higher order thinking, guided reading, and 
culturally responsive teaching; (c) the state develops 
tests which stress extended written performance, but 
within two years abandons these tests, and then, after 
two years of no state testing, issues new tests measur- 
ing recall of knowledge through multiple choice re- 
sponses. In contrast, district policy could assist 
instructional program coherence by requiring elemen- 
tary schools to adopt a research-based model of lit- 
eracy instruction and by supporting extensive 
professional development for school teams to help 
them gain mastery in this approach. 

As indicated in the above examples, any single 
policy could promote instructional program coher- 
ence to a greater or lesser degree. It is important to 
recognize that schools are subject to a mix of policies 
that emanate from different authorities and stakehold- 
ers. This too affects coherence within schools. Critics 
arguing for systemic reform have emphasized the need 
for tighter coordination among local, state, and fed- 
eral policies dealing with everything from assisting 
students with special needs and incentives for school 
improvement, to teacher certification, professional 
development, and accountability mechanisms. Efforts 
to coordinate such policies among different parts of the 
policy system could increase instructional program co- 
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herence within schools if the individual policies them- 
selves reinforce coherence. 

Why Sh®yld lmista©tiS®Dial 
Program Colheireince Pr®m@ft@ 
SMtsmt Aohtowtsmtsnt? 

Several lines of theory and research suggest that in- 
structional program coherence should assist student 
achievement. This can occur 
both by helping teachers 
work more effectively on 
problems of school im- 
provement, and by directly 
increasing student engage- 
ment and learning. 

Assisting Student 
Learning and 
Engagement 

Research on learning and 
cognition indicates that stu- 
dents at all ages are more 
likely to learn when their ex- 
periences connect with and build upon one another. 
To the extent that experiences are disconnected, it is 
more difficult for students to incorporate new un- 
derstanding into prior knowledge and to alter prior 
knowledge when necessary. Studies in cognitive sci- 
ence indicate that learning takes time and requires 
recurring opportunities to practice and to apply 
knowledge and skills in new contexts. Material learned 
through short-term exposure and only in reference to 
a limited context, is less likely to be retained and trans- 
ferred to other settings . 8 For example, students are 
more likely to learn how to use and appreciate frac- 
tions if they see how fractions are applied to a vari- 
ety of problems, such as clocking their running time 
in school races or mixing the right color paint for 
their drawings. 

Compared to disconnected short-term experiences, 
integrated experiences, sustained long enough for suc- 
cessful completion, provide greater clarity about what is 
required for mastery, and how prior knowledge can be 



applied to future questions. Students learning to read, 
for example, are more likely to gain basic skills, and 
the confidence to tackle more challenging tasks, in 
settings where all of their teachers assist their read- 
ing in a consistent manner. In contrast, when there 
is little connection between prior, present, and fu- 
ture activities, and when experiences are too brief 
to allow for mastery, it is more difficult for stu- 
dents to process the information. 

Most of the re- 
search on the impor- 
tance of connected 
learning experiences, 
and the application of 
ideas across multiple 
contexts, comes from 
research on instruc- 
tion within class- 
rooms. But, it is 
reasonable to assume 
that these theories 
and insights also ap- 
ply to learning that 
occurs across differ- 
ent classrooms and to 
learning that occurs as students move from one 
grade to the next. 

Research on motivation suggests that students are 
more likely to engage in the difficult work of learn- 
ing when curricular experiences within classes, among 
classes, and over time are connected to one another. 
As explained above, coherent instruction develops 
competence more effectively than incoherent instruc- 
tion. When children see themselves developing com- 
petence, they are more motivated to work, because 
fulfilling the basic human need for mastery builds con- 
fidence that exerting effort will bring success. In con- 
trast, when faced with incoherent activities, students 
are more likely to feel that they are the target of ap- 
parently random events, and that they have less knowl- 
edge of what should be done to be successful. This 
reduces student engagement in the hard work that 
learning often requires. Thus, incoherent activities un- 
dermine opportunities to gain mastery and the con- 
fidence that motivates further learning . 9 
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Together these points suggest that where curricu- 
lum, instruction, and special programs are coordi- 
nated, one could expect enhanced student 
achievement. Compared to uncoordinated and short- 
term learning experiences, more coherent experiences 
can provide absorbing activities that increase students’ 
motivation to engage in the difficult work of learn- 
ing. They also offer more opportunities for cognitive 
processing, because knowledge and skills previously 
acquired can be used to achieve mastery in new areas. 

Assisting Teacher Effectiveness 

Instructional program coherence might also be ex- 
pected to assist teacher learning and effectiveness. Ac- 
cording to the research on learning and motivation 
summarized above, teachers who participate in co- 
herent professional development experiences, as op- 
posed to short-term, unrelated activities, are also more 
likely to learn from those experiences and to integrate 
that new knowledge into their teaching. In addition, 
research on organizations and effective management 



indicates that professionals who work together on 
integrated activities aimed at clear goals produce 
higher quality goods and services . 10 

There are at least two reasons for the increased 
quality achieved in these settings. First, coordination 
of activity amplifies workers’ access to and use of tech- 
nical resources and expertise. For example, if teachers 
within a grade level pool their knowledge on the most 
effective ways to use cooperative learning in the study 
of mathematical estimation, each teacher has an op- 
portunity to improve his or her skill. Second, con- 
necting the work of different teachers to common 
purposes, activities and practices that are pursued over 
an extended period gives teachers’ work more mean- 
ing, thereby increasing their motivation and commit- 
ment to reach goals. In contrast, when a teacher knows 
from prior experience that ideas and initiatives are 
often introduced and then quickly abandoned, it 
makes little sense to expend much effort to change 
one’s practice. 
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More generally, these points on student learn- 
ing and engagement, and on teacher effectiveness 
are consistent with research on school effectiveness. 
This includes studies on total quality management, 
effective schools, high-capacity schools, Catholic high 
schools, and the differences between public and pri- 
vate schools. Such studies suggest a set of key factors 
that can lead to higher school performance. 11 These 
factors include the importance of a sustained organi- 



zational focus, staff agreement on clear and specific 
goals, more common academic expectations and cur- 
riculum for all students, teacher collaboration and col- 
lective responsibility for meeting goals, and a 
consistent climate of positive supports and high ex- 
pectations for all students and staff. Each of these is 
reflected in one or more of the indicators we propose 
for instructional program coherence. 
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Survey and 
lament Data 

To study the relationship between 
school program coherence and stu- 
dent achievement, we needed mea- 
sures of instructional program 
coherence and the academic 
achievement levels of students 
within those schools. 

Sample 

The Consortium teacher surveys in 
1994 and 1997, which included 
items on instructional program co- 
herence, were distributed to teach- 
ers in all Chicago public 
elementary schools. We merged 
these survey data with the Iowa 
Tests of Basic Skills in Reading and 
Mathematics for all students in 
grades 2 through 8 in these schools. 
Our research used data from the 
222 elementary schools that par- 
ticipated in the surveys in both 
years. This involved 5,358 teach- 
ers in 1994 who had valid re- 
sponses on the coherence items, 
and 5,560 teachers in 1997, with 
an average of 24 teachers per school 
per year in 1994 and 25 teachers 
per school in 1997. The number 
of students who took the tests var- 
ied between grade levels, years, and 
the subjects of reading and math- 
ematics, but the average number 
of students taking the tests in any 
year was 81,493, with an average 
of 367 students per school. 

Measuring Instructional 
Program Coherence 

The survey items used to construct 
measures of teachers perceptions 



of school program coherence are 
listed below. The number of items 
on this topic was expanded in the 
1997 survey over those asked in 
1 994. To allow for a direct compari- 
son of teachers responses from 1994 
to 1 997, we mathematically equated 
common items in the two surveys 
through Item Response Theory, us- 
ing Rasch Rating Scale analysis. 1 The 
1 994 and 1 997 measures were then 
fit to a common 10-point scale. 
Teachers' responses were averaged 
in each school to produce a mea- 
sure of each school's level of in- 
structional program coherence in 
both 1994 and 1997. 2 

Items in the 1994 survey. To 
what extent do you agree or disagree 
with each of the following statements 
(strongly disagree, disagree, agree, 
strongly agree)? 

• You can see real continuity 
from one program to another 
in this school. 

• Many special programs come 
and go in this school (reverse 
scored). 

• Once we start a new program, 
we follow up to make sure that 
it's working. 

• We have so many different pro- 
grams in this school that I can't 
keep track of them all (reverse 
scored). 

Additional Items in the 1997 
Survey. To what extent do you 
agree or disagree with each of the 
following statements (strongly dis- 



agree, disagree, agree, strongly 
agree)? 

• Curriculum, instruction, and 
learning materials are well co- 
ordinated across the different 
grade levels at this school. 

• There is consistency in cur- 
riculum, instruction, and 
learning materials among 
teachers in the same grade 
level at this school. 

• Most changes introduced at 
this school have little relation 
to teachers' and students' real 
needs and interests. 

• Most changes introduced at 
this school help promote the 
school's goals for learning. 

• To what extent have coordi- 
nation and focus of the 
school's instructional pro- 
gram changed in the past two 
years at your school (worse, 
no change, better)? 

Student Achievement 
Measures 

Academic achievement was mea- 
sured through the Iowa Tests of Ba- 
sic Skills (ITBS) in reading and in 
math. For each year, from 1993 to 
1997, we calculated the average 
achievement of students in each of 
the sample schools. Achievement 
scores were computed in logits, 
based on an item response theory 
equating of the different forms and 
test levels of the ITBS used by the 
Chicago Public Schools during this 
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